A Differential Adaptive Learning Rate Method for Back-Propagation Neural Networks

In this paper a high speed learning method using differential adaptive learning rate (DALRM) is proposed. Comparison of this method with other methods such as standard BP, Nguyen-Widrow weight Initialization and Optical BP shows that the network’s learning speed has highly increased. Learning often takes a long time to converge and it may fall into local minimas. One way of escaping from local minima is to use a large learning rate at first and then to gradually reduce this learning rate. In this method which is used in multi-layer networks using back-propagation learning algorithm, network error is reduced in a short time using differential adaptive learning rate. Key-words: standard BP, optical-BP, adaptive learning rate, Nguyen-Widrow method, activation function.